HOST INTERFACE FOR IMAGING ARRAYS 

5 This application claims the benefit of US Provisional Patent Application Serial 

No. 60/177,496 filed on January 21, 2000. 

Field of the Invention 

10 The invention relates generally to integrated electronic image sensing circuitry 

and more particularly to CMOS imaging circuitry. 

Background of the Invention 

15 Integrated circuit (IC) technology, applied to imaging, is revolutionizing that 

field. Semiconductors can be used to represent an image as an electrical signal. 
Charge coupled devices (CCDs) are the most significant commercial IC technology to 
date. However, when compared with CMOS technology, there are many advantages 
to producing CMOS image devices. 

20 

CMOS is a less expensive technology; CMOS employs fewer mask layers and 
is a more mature fabrication technology with greater commercial volume. CCD 
technology complexity causes lower fabrication yield. One of the main .benefits of 
employing CMOS technology, compared to CCD, is the ability to include image- 
25 processing elements on the same substrate as the imaging circuitry. 

On a monolithic semiconductor IC, with a surface coincident to an optical 
focal plane, photosensitive elements are employed in pixels that are arranged in an 
array of rows and columns. The basis for the pixels of CMOS technology is a 
30 photosensitive diode. In an active pixel arrangement each pixel photodiode is buffered 
from the shared readout components by an amplification stage. 



IC image sensors of existing technologies provide video style output. In one 
example, such a sensor receives master clock input. The sensor derives data sample, 
line, and clocks from this master clock. These clocks, which correspond to pixel, row, 
5 and column, control the sampling rate of the imaging array. The pixel data of such a 
sensor is output at the same rate as it is sampled. The derived clocks are output as well 
to synchronize the data output. The result is a stream of synchronized pixel intensities 
comprising a video frame. 

10 This output is incompatible with the data interface of commercial 

microprocessors, without the use of additional glue logic. A commercial 
microprocessor data interface consists of address and control output signals and data 
input/output signals. This configuration allows the processor to randomly access any 
word of data in memory by asserting various addresses. 

15 

In an image acquiring computer system based on such a sensor and such a 
processor, additional interface circuitry to respond to the sensor clock outputs to 
sample the video information, and to make this video data available in the memory 
space of the processor. Optionally, this interface circuit may include interrupt signals 
20 to the processor, and enough memory space for a number of pixels. 

Such additional circuitry diminishes the benefit of a single substrate that 
integrates sensor and processing elements. The CMOS technology optimum cost 
benefit is not reached. 

25 

Therefore, there is a need for an interface which may be integrated with the 
imaging array which a system processor can access to directly receive imaging data. 



Summary of the Invention 

This invention is directed to an interface for receiving data from an image 
5 sensor having an imaging array and a clock generator, and for transferring the data to 
a processor system. The interface comprises a memory for storing the imaging array 
data and the clocking signals at a rate determined by the clocking signals. In response 
to the quantity of data in the memory, a signal generator generates a signal for 
transmission to the processor system and a circuit controls the transfer of the data 
10 from the memory at a rate determined by the processor system. The memory may be 
a first-in first-out (FIFO) buffer or an addressable memory. 

The signal generator may generate an interrupt signal for transmission to the 
processor system or a bus request signal for transmission to a bus arbitration unit for 

1 5 the processor system. The circuit for controlHng the transfer of the data may include a 
command decoder for receiving address and command signals from the processor 
system, a configuration register for storing configuration data for the FIFO buffer and 
a read control for controlling the read-out of the FIFO buffer, and may further include 
a bus command unit for receiving control of the system bus and providing an address 

20 for the data read-out from the memory. 

In accordance with another aspect of this invention, an integrated 
semiconductor imaging circuit for use with an electronic processing system having a 
data bus comprises an imaging array sensor having an array of sensing pixels and an 

25 array address generator integrated on a die and an interface integrated on the same 
die. The interface is adapted to receive data from the imaging array sensor as 
determined by the imaging array and to transfer the data to the electronic processing 
system as determined by the electronic processing system. The interface may include 
a memory such as a FIFO buffer or an addressable memory for storing imaging array 

30 data and address signals at a rate determined by the imaging array sensor, and a circuit 
for controlling the transfer of the data from the memory means to the data bus at a rate 
determined by the electronic processing system. The imaging circuit may further 



include a bus arbitration circuit integrated on the same die and coupled to the circuit 
for controlling the transfer of the data. 

5 In accordance with a further aspect of this invention, an integrated 

semiconductor imaging circuit for use with an electronic processing system having a 
data bus may comprise an imaging array of sensing pixels, a buffer for storing data 
received at an input port and for outputting data through an output port to the data 
bus, a circuit for transferring data from a selected pixel to the buffer input port, a 
10 circuit for determining the quantity of data in the buffer, a circuit for alerting the 
electronic processing system when the quantity of data in the buffer attains a 
predetermined level and a controller adapted to respond to the electronic processing 
system for controlling the transfer of the stored data through the buffer output port. 

15 In accordance with another aspect of this invention, an integrated 

semiconductor imaging circuit for use with an electronic processing system having a 
data bus and a system address/control bus, may comprise an imaging array of sensing 
pixels, a buffer for storing data received at an input port and for outputting data 
through an output port to the data bus, a circuit for transferring data from a selected 

20 pixel to the buffer input port, a circuit for determining the quantity of data in the 
buffer, a controller for seeking control of the data bus when the quantity of data in the 
buffer attains a predetermined level and adapted to respond to the availability of the 
data bus for controlling the transfer of the stored data through the buffer output port. 
The integrated semiconductor imaging circuit may further include a bus arbitration 

25 unit for receiving data bus control requests and for providing data bus control in 
response to a request, and the controller for receiving bus control comprising a 
register for storing and incrementing destination addresses, and a circuit for asserting 
the destination address and write controls on the system address/control bus. 

30 In accordance with a further aspect of this invention, an integrated 

semiconductor imaging circuit for use with an electronic processing system having a 
data bus, may comprise an imaging array of sensing pixels, an addressable memory 



having a plurality of memory cells arranged in rows and columns for storing data 
received at an input port and for outputting data through an output port to the data 
bus, a circuit for transferring data from a selected pixel to a selected memory cell 
5 through the memory input port, a circuit for determining the quantity of data in the 
memory, a circuit for alerting the electronic processing system when the quantity of 
data in the memory attains a predetermined level, and a controller adapted to respond 
to the electronic processing system for controlling the transfer of the stored data 
through the memory output port. 

10 

In accordance with another aspect of this invention, an integrated 
semiconductor imaging circuit for use with an electronic processing system having a 
data bus and a system address/control bus, may comprise an imaging array of sensing 
pixels, an addressable memory having a plurality of memory cells arranged in rows 

15 and columns for storing data received at an input port and for outputting data through 
an output port to the data bus, a circuit for transferring data from a selected pixel to a 
selected memory cell through the memory input port, a circuit for determining the 
quantity of data in the memory, and a controller for seeking control of the data bus 
when the quantity of data in the memory attains a predetermined level and adapted to 

20 respond to the availability of the data bus for controlling the transfer of the stored data 
through the memory output port. The integrated semiconductor imaging circuit may 
further include a bus arbitration unit for receiving data bus control requests and for 
providing data bus control in response to a request, and the controller for receiving 
bus control comprising a register for storing and incrementing destination addresses, 

25 and a circuit for asserting the destination address and write controls on the system 
address/control bus. 

Other aspects and advantages of the invention, as well as the structure and 
operation of various embodiments of the invention, will become apparent to those 
30 ordinarily skilled in the art upon review of the following description of the invention 
in conjunction with the accompanying drawings. 
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Brief Description of the Drawings 

The invention will be described with reference to the accompanying drawings, 
5 wherein: 

Figure 1 is a block diagram of a computer system utilizing the imaging array 

sensor; 

Figure 2 is a block diagram of an imaging array sensor including the interface 
of the present invention; 
10 Figure 3 is a block diagram of the pixel imaging array and access; 

Figure 4 is a block diagram of the video clock and array address generator; 
Figure 5 is a block diagram of a FIFO buffer; 

Figure 6 is a block diagram of a computer system with bus arbitration 
utilizing the imaging array sensor; 
15 Figure 7 is a block diagram of an imaging array sensor that includes an 

interface having bus arbitration circuitry; and 

Figure 8 is a block diagram of an imaging array sensor that includes an 
interface having an addressable memory. 

20 Detailed Description of the Invention 

The imaging computer system illustrated in figure 1 includes a central 
processing unit (CPU) 10, other memory and system components 11, an imaging 
array sensor 12, an interface 13 in accordance with the present invention and a video 

25 clock generator 14. The CPU 10, components 11 and interface 13 all have access to a 
system data bus 15 and are controlled by the CPU 11 via the system control and 
address bus 16. The clock generator 14 provides pixel clock signals Cp to the imaging 
array sensor 12. The interface 13 is further connected to, the CPU 10 through an 
interrupt bus 17 by which the CPU 10 is signalled that data is available for it to 

30 upload. 



In accordance with the present invention, the interface 13 stores data and 
clocking signals from the imaging array sensor 12 in order to free up the CPU 10 for 
other processing. In addition, the full economic and commercial advantage of CMOS 
5 technology may be gained by integrating the interface 13 on the same die as the 
imaging array sensor 12. 

An embodiment of the interface 13 is illustrated as a block diagram in figure 
2. The imaging array sensor 12 includes an imaging array 21 which is an array of 
10 active photosensitive pixels with access control as will be described further with 
reference, to figure 3. The imaging array 21 further includes an array address 
generator 22 which generates the column addresses Ac, the row addresses Ar, the row 
clock Cr and the frame clock Cp as will be described further with reference to figure 
4. 

15 

Referring to figure 3, the array 30 of pixels 33 is organized in rows 31 and 
columns 32. Each pixel 33 is located at the intersection of a row 31 and a column 32. 
The row control lines 34 provide access to a row 31 of pixels 33. The row line 34 is 
driven by the row drivers 35 in response to the row address signal Ar. Each selected 
20 pixel 33 asserts data onto its own column data line 36 when accessed. The data on 
lines 36 is amplified by column amplifiers and second stage amplification in unit 37. 
Unit 37 further selects the column 32 as determined by column address Ac from 
which array data Da is placed on the array output 38. 

25 Referring to figure 4, the array address generator 22 is shown in greater detail. 

The column address Ac is generated by a column counter 41 which is incremented by 
the video system clock Cp. The maximum number of sequential addresses generated 
by the column counter 41 will depend on the number of columns in the imaging array 
21, however the actual number of sequential addresses generated by the column 

30 counter 41 will be determined by the column boundary signal Be which is controlled 
by the CPU 10 as will be described later. The row clock Cr is generated by the 
overflow of the column counter 41. The row counter 42 generates the row address 




signal Ar based on the row clock signal Cr and the row boundary signal Br. The 
maximum number of sequential addresses generated by the row counter 42 will 
depend on the number of rows in the imaging array 21, however the actual number of 
5 sequential addresses generated by the row counter 42 will be determined by the row 
boundary signal Br which is controlled by the CPU 10 as will be described later. The 
row clock Cr is also applied to an output 43 from the array address generator 22. The 
row counter 42 also generates a frame signal Cf based on count overflow. 

10 Referring again to figure 2, the interface 13 includes a memory 44 as well as 

devices 45 to 49 required to support the memory 44. In this particular embodiment, 
memory 44 is a first-in first-out (FIFO) buffer memory. FIFO buffer 44 receives 
array data Da from the imaging array, clocking signals Cp from the video clock 
generator 14 and clocking signals Cr and Cf from the array address generator 22. 

15 FIFO buffer 44 is shown in greater detail in figure 5. The imaging array 21 output 
Da, row clock Cr and frame clock Cf are bundled onto a single bus 51 for storage in 
the buffer 44. The storage components of the FIFO buffer 44 are registers 52 arranged 
as a shift register series 53. Since the total number of valid outputs may vary due to 
the differing rates of storage and access, the bus 51 is connected to each register 52. 

20 An increment/decrement counter 54 is used to count the occurrences of FIFO buffer 
44 writes and FIFO buffer 44 reads. Counter 54 has access to the pixel clock Cp and a 
FIFO read signal Sr. The FIFO counter 54 output Sc is applied to buffer output 55 
and to the Register address decoder 56. The decoder uses the counter output Sc and 
pixel clock Cp in determining when to assert the appropriate register write signal on 

25 lines 57. The read signal Sr is connected to the shift registers 52 to shift the registers 
by a number of registers depending on the read signal Sr value. The same number of 
registers, from the end of the buffer, asserts data Di on the system data bus 15 during 
this operation. 

30 There are basically three types of FIFO buffers, each of which may be used 

with the present invention. The first type of buffer 44 is the one shown in figure 5 
where stored data is removed from buffer register series 53 from the first register 53 
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on the right hand end and data from the bus 51 is written into the last available shift 
register 52 from the left end of the buffer register series 53. A second type of buffer 
is one where the data is written into the first register on the left hand end of the buffer 
5 register series and data is taken out of the buffer register series from the first register 
with data in the series looking at it from the right end of the register series. The third 
type of buffer is one in which data from the data bus is written into the last available 
shift register looking from the left end of the buffer register series and data is taken 
out of the buffer register series from the first register with data in the series looking at 
10 it from the right end of the register series. In all three cases, data is removed from the 
buffer in the same sequence that it is entered into the buffer. 

Referring again to figure 2, the interface 13 includes devices 45 to 49 to 
support the FIFO buffer 44. The devices include a Chip Command Decoder 45, FIFO 
15 Configuration Registers 46, FIFO Read Control, an Interrupt Generator 48 and Array 
Registers 49. 

The CPU 10 accesses the registers 46 and 49 and FIFO buffer 44 through the 
Chip Command Decoder 45 by asserting the necessary read or write commands, along 

20 with the address on the system address and command bus 16. The command decoder 
45 identifies any buffer or register being addressed and asserts the necessary read or 
write signal on the FIFO read control 47 line 56, the FIFO configuration register 46 
command bus 57, or the array register 49 command bus 58. The signal on line 56 
permits the FIFO read control 47 to generate a FIFO read signal Sr in response the 

25 output bus width signal Sew. Variation of the FIFO 44 output bus width register 
provides compatibility with a variety of system bus configurations such as 8-bit or 
32-bit. 

The FIFO configuration registers 46 include the FIFO output bus width, the 
30 FIFO limit value, the FIFO interrupt mask, and the FIFO interrupt register. All of 
these registers are connected to the system data bus 15 and are read/write capable, 
except the FIFO interrupt register, which is read only and determines its value from 



the interrupt generator as signal Si. The reading and writing of these registers is 
controlled by the FIFO register command bus 57. The output of the FIFO 
configuration registers include FIFO limit signal Sl from the FIFO limit register, the 
5 interrupt enable signal Se from the FIFO interrupt mask, and the output bus width 
signal Sbw from the FIFO output bus width register. 

The interrupt generator 48 compares the FIFO counter output Sc and the FIFO 
Hmit Sl- If Sc > Sl and if the interrupt enable signal Se is valid, the generator 48 
10 asserts the interrupt signal Si to the CPU 10 via the interrupt bus 17. The use of an 
interrupt signal Si as an interrupt to the CPU 10 allows the processor to multi-task. It 
performs a buffer 44 unload operation when the interrupt is asserted, and carries out 
other programmed tasks at all other times. 

15 Access to the array registers 49 is controlled by the array register command 

bus, 58, Data is exchanged with the system data bus 15. The content of the registers 
49 defines the number of rows and columns to be employed in the imaging array 21. 
This information is communicated to the array address generator 22 by the row and 
column boundary signals Br and Be. 

20 

The above interface 13 signals the CPU 10 through the interrupt signal Si 
when it has an amount of data approaching the limits of its storage capacity. The 
CPU then responds by having the data downloaded onto the system bus 15. It is 
important for the CPU to respond to the interface faster then the imaging array 21 can 

25 generate data. In addition, the size of the FIFO buffer 44 will also depend on the 
latency of the CPU 10, since during the period of time required by the CPU 10 to 
respond to the interrupt signal Si, data is being stored in the buffer 44. The faster that 
the CPU 10 is able to respond to the interrupt and accept the downloaded data, the 
smaller the buffer 44 can be and the less space that it will require if integrated on the 

30 die with the imaging array 21. However, in real time control applications, it is 
important that the interface 13 and the CPU 10 be matched so that the data from all 
frames scanned by the imaging array 21 is properly and completely transferred to the 



CPU 10. This requirement may be relaxed somewhat for camera type appHcations 
where the necessity of capturing all frames is not required. 

5 In a further embodiment of the present invention as illustrated in figure 6, the 

interface 73 would interact with the CPU 10 and other system components through a 
bus arbitration unit 61. Rather then send an interrupt signal Si to the CPU 10, the 
interface 73 sends a bus request signal Sbr to the bus arbitration unit 61 and receives 
an arbitration acknowledgement signal Saa when the bus 15 is available to it for 
10 downloading data. As illustrated in figure 6, the other units, CPU 10 and components 
11 in the system have their own arbitration request lines 62 and arbitration 
acknowledgement lines 63. The Bus Arbitration Unit 61 receives all the requests for 
the bus 15 and selects one unit that is acknowledged as the current bus master. 

15 The required components in the interface 73 that are required in order for it to 

be compatible with a bus arbitration system are shown in figure 7. A Bus Request 
Generator 64 functions in the same manner as the Interrupt Generator 48 shown in 
figure 2. A bus request signal Sbr is generated in the same manner as the interrupt Si. 
If Sc ^ Sl and the bus request enable signal Sbe is valid, the generator 64 asserts the 

20 bus request signal Sbr to the bus arbitration unit 61. 

An arbitration acknowledge signal Saa notifies the interface 73 that the 
interface 73 may assert command of the bus 15. The arbitration acknowledge signal 
Saa is applied to the chip command decoder 45 and a bus command unit 65. The 

25 arbitration acknowledge signal Saa deactivates the command decoder 45 for the 
duration that the interface controls the bus 15. On receiving the arbitration 
acknowledge signal Saaj the bus command unit 65 will activate an output address unit 
66 via the request output address signal Sar and receive from it the next address on 
the output address signal San- This address is sent out onto the system address and 

30 control line 16. At the same time the bus command unit 65 asserts the necessary read 
or write signal on the FIFO read control 47 line 67, 




The address may represent a location in the CPU 10, however, one advantage 
of this arrangement is that the address may be to a location in one of the system 
components 11 such as a memory so that the data may be stored in the system for 
5 processing by the CPU 10 without the CPU 10 being disturbed to make the transfer. 
The output address unit 66 contains a register and increment circuit for the purpose of 
recording and updating this address. The addresses in the output address unit 66 are 
transferred to the address registers through bus 15 under the control of a signal from 
CPU 10 on the system control and address bus 16 through command decoder 45, 

10 

As stated previously, the imaging array sensor 12 and the interface may be 
integrated onto one die. However, in addition, the Bus Arbitration Unit 61 may also 
be integrated onto the same die, and thus the bus arbitration request and acknowledge 
signals on lines 62 and 63 become external signals of the integrated unit. 

15 

In a further embodiment of the present invention, the memory in the interface 
83 may be an addressable memory 81 as shown on figure 8. For purposes of writing 
to memory 81 from the imaging array 21 the row and frame clocks CR.and Cp serve 
as row and column addresses. The video system clock Cp serves as a write clock. 
20 Thus the memory 81 records the imaging array output Da at the same rate as the 
imaging array 21, and in the same array order as the imaging array 21. 

For reading purposes, the read control signal Sr provides the necessary 
address information, bus width information and read control timing. The memory read 

25 control 82 derives this information from the memory configuration registers 84 via the 
output bus width signal Sbw and from the command decoder 45 via the read enable 
and read address bus 16 and through line 85. The memory configuration registers 84 
are identical to the FIFO configuration registers 46. The memory 81 also includes an 
increment/decrement counter similar to counter 54 to interface with the interrupt 

30 generator 48. In addition, the interface 83 may be adapted for use with a bus 
arbitration unit 61 in the same manner that the interface 73 has been adapted as 
described in conjunction with figure 7. 




Though the use of an addressable memory 81 in interface 83 does not provide 
the size, simpHcity and lower cost of a FIFO memory, the fact that the memory is 
addressable allows the CPU to select parts or patterns from each frame for processing 
5 though the memory 81 would normally hold one frame which would be refreshed 
with each scan. 

While the invention has been described according to what is presently 
considered to be the most practical and preferred embodiments, it must be understood 

10 that the invention is not limited to the disclosed embodiments. Those ordinarily 
skilled in the art will understand that various modifications and equivalent structures 
and functions may be made without departing from the spirit and scope of the 
invention as defined in the claims. Therefore, the invention as defined in the claims 
must be accorded the broadest possible interpretation so as to encompass all such 

15 modifications and equivalent structures and fianctions. 



